Papers with bottom-up and
Long Document Summarization with Top-down and Bottom-up Inference (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent models infer latent representations of words or tokens with a transformer encoder, which is bottom-up and thus does not capture long-distance context well. |
| Approach: | They propose a method to infer latent representations of words or tokens in documents . they assume a hierarchical structure of a document where top-level captures long range dependency . |
| Outcome: | The proposed model can summarize an entire book and achieve competitive performance on a wide range of document summarization benchmarks. |
Improving Constituent Representation with Hypertree Neural Networks (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods of span representation are based on simple derivations from word representations and do not utilize compositional structures of natural language. |
| Approach: | They propose a hypertree neural network that is structured with constituency parse trees to improve representations of constituent spans. |
| Outcome: | The proposed model improves representations of constituent spans using constituency parse trees. |
ISAAQ - Mastering Textbook Questions with Pre-trained Transformers and Bottom-Up and Top-Down Attention (2020.emnlp-main)
Copied to clipboard
| Challenge: | Textbook Question Answering is a complex task that requires reasoning with multimodal information from text and diagrams. |
| Approach: | They propose to use transformer language models and bottom-up and top-down attention to tackle the language and visual understanding challenges of text and diagrams. |
| Outcome: | The proposed system achieves unprecedented accuracies on all TQA question types . the system also obtains state-of-the-art results in other demanding datasets . |